Image-capturing apparatus

ABSTRACT

An image-capturing apparatus according to the present invention includes: an image sensor; a gaze detecting sensor configured to detect a gaze of a user; and at least one memory and at least one processor which function as: an object-detecting unit configured to detect an object from an image captured by the image sensor; and a control unit configured to, if the object-detecting unit detects an object of a specific type in a case where a state of the gaze detecting sensor is a first state for detecting the gaze, change the state of the gaze detecting sensor to a second state in which an electric power consumption of the gaze detecting sensor is less than in the first state.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an image-capturing apparatus, and moreparticularly to a technique for detecting the gaze of a user of theimage-capturing apparatus.

Description of the Related Art

An image-capturing apparatus such as a digital camera is known to havefunctions including a main-object determining function, anobject-tracking function, and a gaze detecting function. The main-objectdetermining function selects (determines) a target main object forautofocusing (AF), tracking, or the like from one or more objectspresent in the captured image. The object-tracking function tracks anobject, and the gaze detecting function detects the gaze of the user ofthe image-capturing apparatus.

Japanese Patent Application Publication No. 2016-48863 discloses atechnique for determining, as the main object, an object that hasperformed a specific action. Japanese Patent Application Publication No.H8-29826 describes a technique for detecting the position to which thegaze of the user of the image-capturing apparatus is directed, detectinga feature of the object present at the detected position, and trackingthe object on the basis of the detected feature. Japanese PatentApplication Publication No. H2-65836 discloses a technique forirradiating the eye with infrared light, capturing an image of thereflected light from the eye, detecting the position of the Purkinjeimage and the center position of the pupil from the captured image, anddetecting the direction of the gaze on the basis of the detectedpositions.

To detect the movement of the gaze with high accuracy, the executioncycle of gaze detection needs to be short. However, a shorter executioncycle of gaze detection increases the power consumption of theimage-capturing apparatus.

SUMMARY OF THE INVENTION

The present invention provides a technique that can reduce the powerconsumption of an image-capturing apparatus capable of detecting thegaze of the user.

The present invention in its first aspect provides an image-capturingapparatus including: an image sensor; a gaze detecting sensor configuredto detect a gaze of a user; and at least one memory and at least oneprocessor which function as: an object-detecting unit configured todetect an object from an image captured by the image sensor; and acontrol unit configured to, if the object-detecting unit detects anobject of a specific type in a case where a state of the gaze detectingsensor is a first state for detecting the gaze, change the state of thegaze detecting sensor to a second state in which an electric powerconsumption of the gaze detecting sensor is less than in the firststate.

The present invention in its second aspect provides an image-capturingapparatus including: an image sensor; a gaze detecting sensor configuredto detect a gaze of a user; and at least one memory and at least oneprocessor which function as: an object-detecting unit configured todetect an object from an image captured by the image sensor; and acontrol unit configured, in a state of the gaze detecting sensor is afirst state for detecting the gaze: to not change the state of the gazedetecting sensor even if the object-detecting unit detects a pluralityof human bodies and at least one ball; and to change the state of thegaze detecting sensor to a second state in which an electric powerconsumption of the gaze detecting sensor is less than in the first stateif the object-detecting unit detects one human body and at least oneball.

The present invention in its third aspect provides a control method ofan image-capturing apparatus including an image sensor and a gazedetecting sensor configured to detect a gaze of a user, the controlmethod including: a step of detecting an object from an image capturedby the image sensor; and a step of, if an object of a specific type isdetected from the image in a case where a state of the gaze detectingsensor is a first state for detecting the gaze, changing the state ofthe gaze detecting sensor to a second state in which an electric powerconsumption of the gaze detecting sensor is less than in the firststate.

The present invention in its fourth aspect provides a control method ofan image-capturing apparatus including an image sensor and a gazedetecting sensor configured to detect a gaze of a user, the controlmethod including: a step of detecting an object from an image capturedby the image sensor; and a step of, in a state of the gaze detectingsensor is a first state for detecting the gaze: not changing the stateof the gaze detecting sensor even if a plurality of human bodies and atleast one ball are detected from the image; and changing the state ofthe gaze detecting sensor to a second state in which an electric powerconsumption of the gaze detecting sensor is less than in the first stateif one human body and at least one ball are detected from the image.

The present invention in its fifth aspect provides a non-transitorycomputer readable medium that stores a program, wherein the programcauses a computer to execute a control method of an image-capturingapparatus including an image sensor and a gaze detecting sensorconfigured to detect a gaze of a user, the control method including: astep of detecting an object from an image captured by the image sensor;and a step of, if an object of a specific type is detected from theimage in a case where a state of the gaze detecting sensor is a firststate for detecting the gaze, changing the state of the gaze detectingsensor to a second state in which an electric power consumption of thegaze detecting sensor is less than in the first state.

The present invention in its sixth aspect provides a non-transitorycomputer readable medium that stores a program, wherein the programcauses a computer to execute a control method of an image-capturingapparatus including an image sensor and a gaze detecting sensorconfigured to detect a gaze of a user, the control method including: astep of detecting an object from an image captured by the image sensor;and a step of, in a state of the gaze detecting sensor is a first statefor detecting the gaze: not changing the state of the gaze detectingsensor even if a plurality of human bodies and at least one ball aredetected from the image; and changing the state of the gaze detectingsensor to a second state in which an electric power consumption of thegaze detecting sensor is less than in the first state if one human bodyand at least one ball are detected from the image.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an image-capturing apparatus according to afirst embodiment;

FIG. 2 is a schematic view of images according to the first embodiment;

FIG. 3 is a flowchart of an operation according to the first embodiment;

FIG. 4 is a schematic view of images according to the first embodiment;

FIG. 5 is a flowchart of an operation according to a second embodiment;

FIG. 6 is a flowchart of an operation according to a third embodiment;

FIG. 7 is a schematic view of an operation according to the thirdembodiment;

FIG. 8 is a block diagram of an image-capturing apparatus according to afourth embodiment; and

FIG. 9 is a flowchart of an operation according to the fourthembodiment.

DESCRIPTION OF THE EMBODIMENTS First Embodiment

Referring to drawings, a first embodiment of the present invention isnow described. The basic operation of an image-capturing apparatusaccording to the first embodiment of the present invention is described.The image-capturing apparatus according to the first embodiment sensesthat the user of the image-capturing apparatus looks into the finder(viewfinder) (the user's eye is in proximity to the finder). Then, in aneye-proximity state (a state in which the user's eye is in proximity tothe finder), the image-capturing apparatus detects the user's gaze usinga gaze detecting unit. Based on the detected gaze, the image-capturingapparatus selects an object to be tracked from the captured image (imageof the current frame). Then, when displaying images of the subsequentframes, the image-capturing apparatus generates an image of a trackingframe surrounding the object to be tracked and superimposes it on thecaptured images.

FIG. 1 is a block diagram schematically showing an example of theinternal structure of an image-capturing apparatus 100 according to thefirst embodiment.

A first optical system 101 has multiple lenses including a movable lenssuch as a focus lens and forms an optical image of the image area on theimaging surface of a first imaging element 102.

A central control unit 122, which may be a central processing unit (CPU)for example, loads a program stored in a read only memory (ROM) 124 intoa random access memory (RAM) 123 and executes the program. The centralcontrol unit 122 implements the functions of the image-capturingapparatus 100 by controlling the operation of each component of theimage-capturing apparatus 100. The ROM 124 may be a rewritablenon-volatile memory, for example, and stores data such as a program thatcan be executed by the central control unit 122, set values, and agraphical user interface (GUI). The RAM 123 may be a system memory usedfor loading a program to be executed by the central control unit 122 andstoring values required during the execution of the program. Althoughomitted in FIG. 1 , the central control unit 122 is connected to andcommunicates with the components of the image-capturing apparatus 100.

The first imaging element 102 may be a CMOS image sensor having a colorfilter of the primary color Bayer arrangement. In the first imagingelement 102, multiple pixels, each having a photoelectric conversionarea, are two-dimensionally arranged. The first imaging element 102converts an optical image formed by the first optical system 101 into anelectric signal group (analog image signal) using the pixels. The analogimage signal is converted into a digital image signal (image data) by anA/D converter of the first imaging element 102 and output. The A/Dconverter may be external to the first imaging element 102.

An evaluation-value generating unit 114 generates, from the image dataobtained from the first imaging element 102, signals and evaluationvalues used for autofocusing (AF), evaluation values used for automaticexposure control (AE), and the like. The evaluation-value generatingunit 114 outputs the generated signals and evaluation values to thecentral control unit 122. The central control unit 122 controls theposition of the focus lens of the first optical system 101 based on thesignal and evaluation value obtained from the evaluation-valuegenerating unit 114, the gaze point (line-of-sight position) obtainedfrom a gaze detection memory 125, and the tracking result obtained froma tracking unit 116. Based on these information pieces, the centralcontrol unit 122 determines the image-capturing conditions (such as theexposure time, aperture value, and ISO sensitivity). Theevaluation-value generating unit 114 may generate a signal or anevaluation value from display image data generated by a postprocessingunit 106 described below.

A first preprocessing unit 103 performs color interpolation on the imagedata obtained from the first imaging element 102. The colorinterpolation is a process of converting each of multiple pixel datapieces (pixel values) forming image data into pixel data having valuesof the R component, the G component, and the B component, and is alsocalled demosaicing. If necessary, the first preprocessing unit 103 mayperform reduction to reduce the number of pixels and thus lower theprocessing load. The first preprocessing unit 103 stores the processedimage data in a display memory 104.

On the image data stored in the display memory 104, a firstimage-correcting unit 105 performs processing such as correction,including white balance correction and shading correction, andconversion from the RGB format to the YUV format. The firstimage-correcting unit 105 outputs the processed image data to thepostprocessing unit 106. The first image-correcting unit 105 may performcorrection using the image data of one or more frames different from theframe to be processed among the image data stored in the display memory104. For example, the first image-correcting unit 105 may performcorrection using the image data of one or more frames before and/orafter the frame to be processed in a time series.

The postprocessing unit 106 generates recording image data and displayimage data from the image data obtained from the first image-correctingunit 105. For example, the postprocessing unit 106 performs coding onthe image data and generates a data file storing the coded image data asrecording image data. The postprocessing unit 106 outputs the recordingimage data to a recording unit 107. The display image data is image datato be displayed on a display unit 118, and has a size corresponding tothe display size on the display unit 118. The postprocessing unit 106outputs the display image data to an information-superimposing unit 117.

The recording unit 107 records the recording image data obtained fromthe postprocessing unit 106 on a recording medium 108. The recordingmedium 108 may be a semiconductor memory card, an embedded non-volatilememory, or the like.

A second preprocessing unit 109 performs color interpolation(demosaicing) on the image data obtained from the first imaging element102. If necessary, the second preprocessing unit 109 may performreduction to reduce the number of pixels and thus lower the processingload. The second preprocessing unit 109 stores the processed image datain a tracking memory 110. The tracking memory 110 and the display memory104 may have different address spaces in the same memory. The first andsecond preprocessing units 103 and 109 may be the same (common)preprocessing unit.

On the image data stored in the tracking memory 110, a secondimage-correcting unit 130 performs processing such as correction,including white balance correction and shading correction, andconversion from the RGB format to the YUV format. The secondimage-correcting unit 130 stores the processed image data in thetracking memory 110. The second image-correcting unit 130 may performimage processing for obtaining image data suitable for object detection.For example, when the representative brightness of image data (forexample, the average brightness of all pixels) is less than or equal toa predetermined threshold, the second image-correcting unit 130 maymultiply all pixel data pieces of the image data by the same coefficient(gain) so as to obtain image data with a representative brightnesshigher than the predetermined threshold. The second image-correctingunit 130 may perform correction using the image data of one or moreframes different from the frame to be processed among the image datastored in the tracking memory 110. For example, the secondimage-correcting unit 130 may perform correction using the image data ofone or more frames before and/or after the frame to be processed in atime series.

It should be noted that the components concerning the object-trackingfunction, such as the second preprocessing unit 109 and the secondimage-correcting unit 130, do not have to operate when theobject-tracking function is not performed. The image data to which theobject-tracking function is applied is moving image data that iscaptured for live view display or recording, and has a predeterminedframe rate of 30 fps, 60 fps, or 120 fps, for example.

A first detection unit 112 detects at least one area of an object of apredetermined type (object area) from the image data of one frame storedin the tracking memory 110 by the second image-correcting unit 130. Thefirst detection unit 112 stores the detection result of the object area(object-detection result) in the tracking memory 110. For example, anobject-detection result may include the following information for eachobject area. Additionally, an object-detection result may include thenumber of detected object areas for each type of object (object classdescribed below).

-   -   Position and size of the object area    -   Object class representing the type of the object (such as        automobile, airplane, train, bird, insect, human body, head,        eye, face, cat, dog, flower, fish, ball)    -   Object state information 1 indicating whether an eye is detected        in the object area    -   Object state information 2 indicating whether a face and/or head        is detected in the object area

Object state information 2 may indicate the face orientation such as“forward”, “backward”, and “sideways”. The object-detection result mayinclude the reliability of the information included in theobject-detection result. Since a head, face, eye, and the like are partsof a human body, cat, dog, and the like, they can be considered aspartial objects. The object classes “head”, “face”, “eye”, and the likemay be considered as partial object classes. The first detection unit112 may detect a wearable object such as a helmet as the object thatwears the wearable object. For example, the first detection unit 112 maydetect a helmet as a head.

The first detection unit 112 can detect object areas using a knowntechnique for detecting feature areas such as a face area of a person oran animal. For example, a class classifier trained using training datamay be used as the first detection unit 112. There is no limitation tothe identification (classification) algorithm. The first detection unit112 may be obtained by training a classifier that implements multi-classlogistic regression, a support vector machine, a random forest, a neuralnetwork, or the like.

A second optical system 119 has an eyepiece and an optical-path dividingprism. The second optical system 119 sets the eye of the user (the userof the image-capturing apparatus 100) looking into the eyepiece in theimage area, separates the infrared light in the image area with theoptical-path dividing prism, and forms an optical image of the separatedinfrared light on the imaging surface of a second imaging element 120.An image including the user's eye is thus captured. The second opticalsystem 119 also projects the image displayed on the display unit 118onto the retina of the user looking into the eyepiece. As such, thesecond optical system 119 also functions as a finder, allowing the userto visually perceive the image displayed on the display unit 118.

An eye-proximity sensing unit 128 (eye-proximity senor) is arrangedadjacent to the finder (the second optical system 119) together with athird optical system 127, and senses the presence of an eye in proximityto the finder, assuming that an object approaching the finder is theuser's eye. The eye-proximity sensing unit 128 has a secondinfrared-light irradiating unit 129 and an infrared-light sensing unit131.

The second infrared-light irradiating unit 129 may be an infrared-lightemitting diode and irradiates the user (the eye in proximity to thefinder; the eye looking into the finder) with infrared light. When thereis a shielding object such as an eye near the finder, the infrared lightemitted from the second infrared-light irradiating unit 129 passesthrough the third optical system 127, is applied to and reflected on theshielding object, and is then incident on the infrared-light sensingunit 131 through the third optical system 127.

The infrared-light sensing unit 131 senses the incident infrared lightand calculates the distance between the eye-proximity sensing unit 128and the shielding object from the time difference between the time whenthe second infrared-light irradiating unit 129 emits the infrared lightand the time when the infrared-light sensing unit 131 senses theinfrared light. The eye-proximity sensing unit 128 determines whetherthe eye is in proximity (whether the user is looking into the finder)according to the calculated distance. The infrared-light sensing unit131 notifies a gaze detecting unit control unit 115 of information as towhether the eye is in proximity (determination result).

The gaze detecting unit control unit 115 controls the operation of thegaze detecting unit, which detects the user's gaze. For example, thegaze detecting unit control unit 115 receives a notification from theeye-proximity sensing unit 128 as to whether the eye is in proximity.When the eye is in proximity, the gaze detecting unit control unit 115enables the operation of the gaze detecting unit so that gaze detection(operation of detecting the user's gaze) is performed. When the eye isnot in proximity, the gaze detecting unit control unit 115 disables(stops) the operation of the gaze detecting unit so that gaze detectionis not performed. The gaze detecting unit (gaze detecting sensor) mayinclude a first infrared-light irradiating unit 111 (irradiationmember), the second imaging element 120 (image sensor), a thirdpreprocessing unit 121 (processor), and a second detection unit 126(detector), for example.

The first infrared-light irradiating unit 111 may be an infrared-lightemitting diode and irradiates the user (the eye in proximity to thefinder; the eye looking into the finder) with infrared light. When theuser is looking into the eyepiece of the second optical system 119, theinfrared light emitted from the first infrared-light irradiating unit111 is reflected on the user's eye looking into the eyepiece. Theinfrared light reflected on the eye is then separated by an optical-pathdividing prism of the second optical system 119 and forms an image onthe second imaging element 120.

For example, the second imaging element 120 may be a CMOS image sensorhaving a color filter that transmits infrared light. In the secondimaging element 120, multiple pixels, each having a photoelectricconversion area, are two-dimensionally arranged. The optical imageformed by the second optical system 119 by separating infrared lightwith the optical-path dividing prism is converted by the second imagingelement 120 into an electric signal group (analog image signal) usingthe pixels. The analog image signal is converted into a digital imagesignal (image data) by an A/D converter of the second imaging element120 and output. The A/D converter may be external to the second imagingelement 120. When the user is looking into the eyepiece of the secondoptical system 119, the second imaging element 120 captures image dataof the user (specifically, the user's eye looking into the eyepiece).

On the image data obtained from the second imaging element 120, thethird preprocessing unit 121 performs processing, such as filtering forremoving a signal component having a specific spatial frequency band,and reduction for reducing the number of pixels. The third preprocessingunit 121 stores the processed image data in the gaze detection memory125.

The second detection unit 126 detects the user's gaze based on the imagecaptured by the second imaging element 120. Various conventionaltechniques can be used to detect the gaze. For example, the seconddetection unit 126 reads out the image data stored in the gaze detectionmemory 125, and detects the Purkinje image and the iris from the readimage data. The Purkinje image is an image of the infrared light emittedfrom the first infrared-light irradiating unit 111 and reflected on thecornea, and is also called a corneal reflection image. The firstinfrared-light irradiating unit 111 has multiple infrared light sources(for example, four infrared-light emitting diodes), and the seconddetection unit 126 detects multiple Purkinje images each correspondingto one of the infrared light sources. Then, the second detection unit126 detects the user's gaze based on the positions of the Purkinjeimages and the center position of the iris, and stores the gazedetection result in the gaze detection memory 125. For example, based onthe positions of the Purkinje images and the center position of theiris, the second detection unit 126 calculates the position on thedisplay unit 118 at which the user is gazing. Then, the second detectionunit 126 converts the calculated position into a position on the imagedata, and stores the converted position in the gaze detection memory 125as the gaze point, which is the gaze detection result. As the gazedetection result, other information may be stored such as the positionbefore conversion and the angle indicating the direction of the gaze.

A target-determining unit 113 determines, as the tracking target, one ofthe one or more object areas detected by the first detection unit 112.For example, the target-determining unit 113 reads out the gaze pointstored in the gaze detection memory 125, and determines the object areaclosest to the gaze point as the tracking target. The target-determiningunit 113 may determine whether the object class of the object area is aspecific object class. Then, the target-determining unit 113 maydetermine an object area of a specific object class (object of aspecific type) as the tracking target without using the gaze point. Thespecific object class (specific type) will be described below. Whenthere are multiple object areas of specific object classes, thetarget-determining unit 113 may determine, as the tracking target, theobject area closest to the gaze point among the multiple object areas.The target-determining unit 113 stores, as tracking target information,information indicating the object area determined as the tracking targetin the tracking memory 110.

Based on the tracking target information stored in the tracking memory110, the tracking unit 116 estimates, as the tracking area, an areacorresponding to the object area of the tracking target from the imagedata of the frame to be processed (current frame) (tracking process oftracking the object (area)). There is no limitation to the method forestimating the tracking area. For example, the image data of the currentframe and the image data of a past frame captured before the currentframe (e.g., the last frame) are used to estimate, as the tracking area,the area of the current frame corresponding to the object area of thetracking target. Then, the tracking unit 116 obtains the position andsize of the estimated tracking area as the tracking result, and outputsthe tracking result to the central control unit 122 and theinformation-superimposing unit 117.

Here, the object area of the current frame determined as the trackingtarget by the target-determining unit 113 is not the area of the currentframe estimated by the tracking unit 116. The tracking unit 116estimates the area of the current frame corresponding to the object areaof a past frame determined by the target-determining unit 113 as thetracking target. The tracking target information obtained by thetarget-determining unit 113 by determining an object area of the currentframe as the tracking target is used when the tracking unit 116 performstracking processing for the next frame.

The target-determining unit 113 may update the tracking targetinformation for each frame, or it may not update the tracking targetinformation until another object area is determined as the trackingtarget and then update the tracking target information when anotherobject area is determined as the tracking target. When the tracking unit116 performs tracking processing based on the similarity with the colorcomposition information of the object area of the tracking target, thecolor composition information of the object area of the tracking targetmay be updated to the color composition information of the estimatedtracking area to be used for the tracking processing of the next frame.The tracking unit 116 may estimate the tracking area by pattern matchingusing the object area of the tracking target as the template. Thetracking unit 116 may estimate the position and size of the trackingarea using a trained multi-layer neural network including convolutionallayers.

The information-superimposing unit 117 generates an image of thetracking frame based on the size of the tracking area estimated by thetracking unit 116. For example, the tracking frame image may be an imageof a rectangular frame circumscribing the tracking area. Based on theposition of the tracking area, the information-superimposing unit 117generates composite image data by performing composition on the displayimage data obtained from the postprocessing unit 106 to superimpose thetracking frame image so that the tracking frame is displayed at theposition of the tracking area. The information-superimposing unit 117may generate an image representing the current set values and the stateof the image-capturing apparatus 100, and superimpose the generatedimage through composition so that the generated image is displayed at apredetermined position. The information-superimposing unit 117 outputsthe composite image data to the display unit 118.

The display unit 118 may be a liquid crystal display or an organic ELdisplay, for example, and displays an image based on the composite imagedata output by the information-superimposing unit 117.

By the series of processes described above, the live view display forone frame is performed.

Here, the gaze detecting unit control unit 115 reads theobject-detection result stored by the first detection unit 112 in aneye-proximity state from the tracking memory 110, and further controlsthe gaze detecting unit based on the read object-detection result.

FIG. 2 shows an example of images captured by the first imaging element102 and displayed on the display unit 118, gaze points, tracking frame,AF point, and the like. The AF point is the position where autofocusingis performed.

An image 200 is an image captured by the first imaging element 102 anddisplayed on the display unit 118. The image 200 shows a human body 201and the face 202 of the human body 201. The image 200 also shows theleft eye 203 and the right eye 204 of the face 202. From the image 200stored in the tracking memory 110, the first detection unit 112 detectsan area circumscribing the human body 201 as an object area. Then, thefirst detection unit 112 generates information on the object area of thehuman body 201, such as the position and size of the object area of thehuman body 201, object class “human body”, object state information 1“with eye”, and object state information 2 “with face and head”. Thefirst detection unit 112 also detects an area circumscribing the face202 as an object area from the image 200. Then, the first detection unit112 generates information on the object area of the face 202, such asthe position and size of the object area of the face 202, object class“face”, object state information 1 “with eye”, and object stateinformation 2 “with face and head”. The first detection unit 112 alsodetects an area circumscribing the left eye 203 as an object area fromthe image 200 and generates information on the object area of the eye203, such as the position and size of the object area of the eye 203 andobject class “eye”. The first detection unit 112 also detects an areacircumscribing the right eye 204 as an object area from the image 200and generates information on the object area of the eye 204, such as theposition and size of the object area of the eye 204 and the object class“eye”. The first detection unit 112 stores the object-detection resultincluding the information on these object areas in the tracking memory110. The target-determining unit 113 determines the tracking targetbased on the object-detection result stored in the tracking memory 110and the gaze point stored in the gaze detection memory 125 by the seconddetection unit 126. Here, it is assumed that left eye has the highestpriority to be determined as the tracking target, followed by right eye,face, and human body in this order. Accordingly, the target-determiningunit 113 determines the left eye 203 (specifically, the object area ofthe eye 203) as the tracking target for the next and subsequent frames.

An image 210 is an image captured multiple frames after the image 200.Since it is after multiple frames, the position and size of the humanbody 211 in the image 210 are different from those of the human body 201in the image 200. The tracking unit 116 performs pattern matching usingthe area of the left eye 203, which is determined by thetarget-determining unit 113 as the tracking target for a past frame(frame of the image 200), as the template and estimates the area of theleft eye in the image 210 (tracking area). The information-superimposingunit 117 generates a tracking frame 215 based on the tracking result(the position and size of the tracking area). Then, the central controlunit 122 determines the AF point based on the tracking result, which isoutput by the tracking unit 116, and the gaze point, which is stored inthe gaze detection memory 125 by the second detection unit 126. Here, itis assumed that the position of left eye (tracking result) has thehighest priority to be determined as the AF point, followed by theposition of right eye (tracking result), the position of face (trackingresult), the gaze point, and the position of human body (trackingresult) in this order. Accordingly, the central control unit 122determines, as the AF point, the center position of the tracking area(the center position of the tracking frame 215) estimated as the area ofthe left eye, and performs focus-lens control based on the calculationresult on the distance to the target object at the AF point. There is nolimitation to the priority to be determined as the AF point. Forexample, the priority of the face position may be lower than thepriority of the gaze point. In this case, when the eye is not tracked,the gaze point 216 is determined as the AF point.

An image 220 is an image captured multiple frames after the image 210.Since it is after multiple frames, the face of the human body 221 in theimage 220 is hidden behind a tree 225. It is assumed that the trackingunit 116 failed to perform tracking of the eye 203 (pattern matchingusing the area of the eye 203 as the template) for the frame immediatelybefore the image 220, and the target-determining unit 113 has reset thetracking target to the human body. Thus, in the frame of the image 220,the area estimated by the tracking unit 116 (tracking area) is the areaof the human body 221. The information-superimposing unit 117 generatesa tracking frame 222 based on the tracking result (the position and sizeof the tracking area). The central control unit 122 then determines theAF point. Here, it is assumed that the position of left eye (trackingresult) has the highest priority to be determined as the AF point,followed by the position of right eye (tracking result), the position offace (tracking result), the gaze point, and the position of human body(tracking result) in this order. Accordingly, the central control unit122 determines a position 223 that is the same as the gaze point 224 asthe AF point, and performs focus-lens control based on the calculationresult on the distance to the target object at the AF point.

Here, based on the object-detection result obtained by the firstdetection unit 112, the gaze detecting unit control unit 115 controls toenable or disable the operation of the gaze detecting unit. For example,when a position (tracking result) is obtained that has a higher priorityto be determined as the AF point than the gaze point, the gaze point isnot determined as the AF point. Thus, the gaze detecting unit controlunit 115 disables the operation of the gaze detecting unit to reduce thepower consumption of the image-capturing apparatus 100.

FIG. 3 is a flowchart showing an example of an operation of theimage-capturing apparatus 100. FIG. 3 shows an example of an operationof controlling the gaze detecting unit based on the object-detectionresult.

At step S301, based on the object-detection result, the gaze detectingunit control unit 115 determines whether an object of a specific type isdetected, specifically, whether an object area of a specific objectclass is detected. The gaze detecting unit control unit 115 proceeds tostep S302 when an object area of a specific object class is detected,and proceeds to step S312 when an object area of a specific object classis not detected. A specific object class is an object class of apredetermined candidate for tracking (candidate for tracking target) andmay be an object class representing an object that can move. In thisexample, the specific object classes do not include partial objectclasses such as “head”, “face”, and “eye”. Examples of an object thatcan move include a mammal (e.g., a human body, a cat, a dog, a horse), abird, a reptile, a fish, a crustacean, an insect, and a vehicle (e.g., arailroad vehicle, an automobile (e.g., a four-wheeled vehicle, atwo-wheeled vehicle), an aircraft, a ship). Examples of a specificobject class include “human body”, “cat”, “dog”, “bird”, “horse”, “otheranimal”, “reptile”, “fish”, “crustacean”, “insect”, “train”,“automobile”, “motorcycle”, “airplane”, “helicopter”, “ship”, and “othervehicle”.

The gaze detecting unit control unit 115 may proceed to step S306 whenan object area of a specific object class is detected. In this case, theobject area of the specific object class may be determined as thetracking target. When multiple object areas of specific object classesare detected, one of these object areas may be selected and determinedas the tracking target. There is no limitation to the method forselecting an object area. For example, one of the object areas may berandomly selected, or one of the object areas may be selected based onparameters such as the size and position of each object area. The gazedetecting unit control unit 115 may proceed to step S307 when an objectarea of a specific object class is not detected.

At step S302, the gaze detecting unit control unit 115 determineswhether the number of object areas of the object class having thehighest predetermined tracking priority among the detected object areas(the object areas of specific object classes) is two or more. Thetracking priority is the priority to be determined as the trackingtarget. The gaze detecting unit control unit 115 proceeds to step S303when the number is two or more, and proceeds to step S308 when thenumber is one. Here, it is assumed that “human body” has the highesttracking priority, followed by “dog”, “cat”, “bird”, “horse”, “otheranimal”, “automobile”, “motorcycle”, “train”, “airplane”, “helicopter”,“ship”, “other vehicle”, “fish”, “insect”, “reptile”, and “crustacean”in this order. For example, when one object area of the object class“dog” and two object areas of the object class “cat” are detected, thegaze detecting unit control unit 115 proceeds to step S308 because theobject class having the highest tracking priority is “dog” and only oneobject area of the object class “dog” is detected.

It should be noted that the user can freely change the tracking priority(the order of the object classes in terms of the tracking priority). Forexample, the tracking priority may be changed according to theimage-capturing mode of the image-capturing apparatus 100, such as aperson-priority mode, an animal-priority mode, and a vehicle-prioritymode. In the animal-priority mode, the tracking priority is changed suchthat animal tracking is prioritized. For example, “dog” has the highesttracking priority, followed by “cat”, “bird”, “horse”, “other animal”,“human body”, “automobile”, “motorcycle”, “train”, “airplane”,“helicopter”, “ship”, “other vehicle”, “fish”, “insect”, “reptile”, and“crustacean” in this order. That is, “dog”, “cat”, “bird”, “horse”, and“other animal” have higher priority than “human body”. To enable theswitching of orders of “dog” and “cat”, image-capturing modes mayinclude a dog-priority mode and a cat-priority mode, for example. Thenumber of object areas of the object class having the highest trackingpriority may be determined from the object-detection result of one frameor may be determined from the average of the object-detection results ofmultiple frames. The gaze detecting unit control unit 115 may proceed tostep S306 when the number of object areas of the object class having thehighest tracking priority is one. In this case, the object area of theobject class having the highest tracking priority may be determined asthe tracking target. The gaze detecting unit control unit 115 mayproceed to step S307 when the number of object areas of the object classhaving the highest tracking priority is two or more.

At step S303, the gaze detecting unit control unit 115 focuses on themultiple object areas of the object class having the highest trackingpriority. The gaze detecting unit control unit 115 determines whether anobject area of the partial object class “eye” is detected in only one ofthe multiple object areas. When an object area of the partial objectclass “eye” is detected in only one of the multiple object areas, thegaze detecting unit control unit 115 proceeds to step S306. In thiscase, the object area of the partial object class “eye” detected in theobject area of the object class having the highest tracking priority isdetermined as the tracking target. When object areas of the partialobject class “eye” are detected in two or more of the multiple objectareas, the gaze detecting unit control unit 115 proceeds to step S304.When an object area of the partial object class “eye” is not detected inany of the multiple object areas, the gaze detecting unit control unit115 also proceeds to step S304. When the object class having the highesttracking priority represents the type of an object with which an eye isnot to be detected, the gaze detecting unit control unit 115 alsoproceeds to step S304.

A partial object class is an object class that represents the type of apartial object (a part of an object). For example, from object areas ofthe object classes “human body”, “dog”, “cat”, “bird”, “horse”, and“other animal”, object areas of the partial object classes “eye”,“head”, and “face” may be detected. From object areas of the objectclasses “fish”, “insect”, “reptile”, and “crustacean”, object areas ofthe partial object class “eye” may be detected. From an object area ofthe object class “motorcycle”, an object area of the partial objectclass “head (occupant's helmet)” may be detected. From an object area ofthe object class “automobile”, an object area of the partial objectclass “head (occupant's head)” may be detected. These partial objectclasses have higher tracking priority than object classes that are notpartial object classes.

The gaze detecting unit control unit 115 may proceed to step S307instead of proceeding to step S304. The object-detection result of oneframe or the object-detection results of multiple frames may be used todetermine whether an object area of the partial object class “eye” isdetected. For example, when a state in which an object area of thepartial object class “eye” is detected only in the same single objectarea continues for multiple frames, the process may proceed to stepS306. If not, the process may proceed to step S304.

At step S304, the gaze detecting unit control unit 115 again focuses onthe multiple object areas of the object class having the highesttracking priority. The gaze detecting unit control unit 115 determineswhether an object area of the partial object class “face” and an objectarea of the partial object class “head” are detected in only one of themultiple object areas. When object areas of the partial object classes“face” and “head” are detected in only one of the multiple object areas,the gaze detecting unit control unit 115 proceeds to step S306. In thiscase, the object area of the partial object class “face” or “head”detected from the object area of the object class having the highesttracking priority is determined as the tracking target. When objectareas of the partial object classes “face” and “head” are detected intwo or more of the multiple object areas, the gaze detecting unitcontrol unit 115 proceeds to step S305. When an object area of thepartial object class “face” is not detected in any of the multipleobject areas, the gaze detecting unit control unit 115 also proceeds tostep S305. When an object area of the partial object class “head” is notdetected in any of the multiple object areas, the gaze detecting unitcontrol unit 115 also proceeds to step S305. When the object classhaving the highest tracking priority represents the type of an objectwith which a face is not to be detected, the gaze detecting unit controlunit 115 also proceeds to step S305. When the object class having thehighest tracking priority represents the type of an object with which ahead is not to be detected, the gaze detecting unit control unit 115also proceeds to step S305.

The gaze detecting unit control unit 115 may proceed to step S307instead of proceeding to step S305. The determination at step S304 maybe replaced with the determination as to whether an object area of thepartial object class “face” is detected. The determination at step S304may be replaced with the determination as to whether an object area ofthe partial object class “head” is detected. In this case, even if anobject area of the partial object class “face” is not detected, anobject area of the partial object class “head” may be determined as thetracking target. The determination at step S304 may be performed basedon the object-detection result of one frame, or may be performed basedon the object-detection results of multiple frames. For example, when astate in which object areas of the partial object classes “face” and“head” are detected only in the same single object area continues formultiple frames, the process may proceed to step S306. If not, theprocess may proceed to step S305.

At step S305, the gaze detecting unit control unit 115 again focuses onthe multiple object areas of the object class having the highesttracking priority. The gaze detecting unit control unit 115 determineswhether an object area of a partial object class that is not “eye”,“face”, or “head” is detected in only one of the multiple object areas.When an object area of a partial object class that is not “eye”, “face”,or “head” is detected in only one of the multiple object areas, the gazedetecting unit control unit 115 proceeds to step S306. In this case, theobject area of the partial object class that is not “eye”, “face”, or“head” detected from the object area of the object class having thehighest tracking priority is determined as the tracking target. Whenobject areas of one or more partial object classes that are not “eye”,“face”, or “head” are detected in two or more of the multiple objectareas, the gaze detecting unit control unit 115 proceeds to step S307.When an object area of a partial object class that is not “eye”, “face”,or “head” is detected in none of the multiple object areas, the gazedetecting unit control unit 115 also proceeds to step S307. When theobject class having the highest tracking priority represents the type ofan object with which a partial object that is not an eye, face, or headis not to be detected, the gaze detecting unit control unit 115 alsoproceeds to step S307.

The gaze detecting unit control unit 115 may proceed to step S311instead of proceeding to step S307. The object-detection result of oneframe or the object-detection results of multiple frames may be used todetermine whether an object area of a partial object class that is not“eye”, “face”, or “head” is detected. For example, when a state in whichan object area of a partial object class is detected only in the samesingle object area continues for multiple frames, the process mayproceed to step S306. If not, the process may proceed to step S307.

At step S308, the gaze detecting unit control unit 115 determineswhether an object area of the partial object class “eye” is detected inthe object area of the object class having the highest trackingpriority. When an object area of the partial object class “eye” isdetected in the object area of the object class having the highesttracking priority, the gaze detecting unit control unit 115 proceeds tostep S306. In this case, the object area of the partial object class“eye” detected in the object area of the object class having the highesttracking priority is determined as the tracking target. When an objectarea of the partial object class “eye” is not detected in the objectarea of the object class having the highest tracking priority, the gazedetecting unit control unit 115 proceeds to step S309. When the objectclass having the highest tracking priority represents the type of anobject with which an eye is not to be detected, the gaze detecting unitcontrol unit 115 also proceeds to step S309.

At step S308, in the same manner as step S303, the object-detectionresult of one frame or the object-detection results of multiple framesmay be used to determine whether an object area of the partial objectclass “eye” is detected. The gaze detecting unit control unit 115 mayproceed to step S307 instead of proceeding to step S309.

At step S309, the gaze detecting unit control unit 115 determineswhether an object area of the partial object class “face” and an objectarea of the partial object class “head” are detected in the object areaof the object class having the highest tracking priority. When objectareas of the partial object classes “face” and “head” are detected inthe object area of the object class having the highest trackingpriority, the gaze detecting unit control unit 115 proceeds to stepS306. In this case, the object area of the partial object class “face”or “head” detected from the object area of the object class having thehighest tracking priority is determined as the tracking target. When anobject area of the partial object class “face” is not detected in theobject area of the object class having the highest tracking priority,the gaze detecting unit control unit 115 proceeds to step S310. When anobject area of the partial object class “head” is not detected in theobject area of the object class having the highest tracking priority,the gaze detecting unit control unit 115 also proceeds to step S310.When the object class having the highest tracking priority representsthe type of an object with which a face is not to be detected, the gazedetecting unit control unit 115 also proceeds to step S310. When theobject class having the highest tracking priority represents the type ofan object with which a head is not to be detected, the gaze detectingunit control unit 115 also proceeds to step S310.

As with the determination at step S304, the determination at step S309may be replaced with the determination as to whether an object area ofthe partial object class “face” is detected. The determination at stepS309 may be replaced with the determination as to whether an object areaof the partial object class “head” is detected. The determination atstep S309 may be performed based on the object-detection result of oneframe, or may be performed based on the object-detection results ofmultiple frames. The gaze detecting unit control unit 115 may proceed tostep S307 instead of proceeding to step S310.

At step S310, the gaze detecting unit control unit 115 determineswhether an object area of a partial object class that is not “eye”,“face”, or “head” is detected in the object area of the object classhaving the highest tracking priority. When an object area of a partialobject class that is not “eye”, “face”, or “head” is detected in theobject area of the object class having the highest tracking priority,the gaze detecting unit control unit 115 proceeds to step S306. In thiscase, the object area of the partial object class that is not “eye”,“face”, or “head” detected from the object area of the object classhaving the highest tracking priority is determined as the trackingtarget. When an object area of a partial object class that is not “eye”,“face”, or “head” is not detected in the object area of the object classhaving the highest tracking priority, the gaze detecting unit controlunit 115 proceeds to step S311. When the object class having the highesttracking priority represents the type of an object with which a partialobject that is not an eye, face, or head is not to be detected, the gazedetecting unit control unit 115 also proceeds to step S311.

As with the determination at step S305, the determination at step S310may be performed based on the object-detection result of one frame, ormay be performed based on the object-detection results of multipleframes. The gaze detecting unit control unit 115 may proceed to stepS307 instead of proceeding to step S311.

At step S312, the gaze detecting unit control unit 115 calculates thetotal number of detected object areas of each object class (that is, thetotal number of detected object areas) and determines whether thecalculated number is one. The gaze detecting unit control unit 115proceeds to step S311 when the calculated number is one, and proceeds tostep S307 when the calculated number is two or more or 0 (zero).

The gaze detecting unit control unit 115 may proceed to step S311 whenthe calculated number is two or more. The total number of object areasmay be determined from the object-detection result of one frame, or maybe determined from the average of the object-detection results ofmultiple frames.

At step S311, the gaze detecting unit control unit 115 focuses on oneobject area. When the process proceeds from step S310 to step S311, thegaze detecting unit control unit 115 focuses on the object area of thespecific object class. When the process proceeds from step S312 to stepS311, the gaze detecting unit control unit 115 focuses on the objectarea of an object class that is not a specific object class. The gazedetecting unit control unit 115 determines whether the size of thefocused object area is greater than or equal to a predeterminedthreshold. The gaze detecting unit control unit 115 proceeds to stepS307 when the size of the focused object area is greater than or equalto the predetermined threshold, and proceeds to step S306 when the sizeof the focused object area is less than the predetermined threshold.When the process proceeds to step S306, the object area focused at stepS311 is determined as the tracking target.

As described above, at step S305, the gaze detecting unit control unit115 may proceed to step S311 instead of proceeding to step S307. In thiscase, the gaze detecting unit control unit 115 focuses on one of themultiple object areas of a specific object class. At step S312, the gazedetecting unit control unit 115 may proceed to step S311 when the totalnumber of detected object areas is two or more. In this case, the gazedetecting unit control unit 115 focuses on one of the detected multipleobject areas. There is no limitation to the method for selecting theobject area to be focused. For example, one of the object areas may berandomly selected, or one of the object areas may be selected based onparameters such as the size and position of each object area. Theobject-detection result of one frame or the object-detection results ofmultiple frames may be used to determine whether the size of the focusedobject area is greater than or equal to the predetermined threshold. Forexample, when a state in which the size of the focused object area isless than the predetermined threshold continues for multiple frames, theprocess may proceed to step S306. If not, the process may proceed tostep S307.

Furthermore, as the threshold to be compared with the size of thefocused object area, a different threshold may be predetermined for eachobject class. In one example, an image stored in the tracking memory 110(a captured image) may be 160 pixels in the horizontal direction and 120pixels in the vertical direction. In this case, a size of 40 pixels inthe horizontal direction×30 pixels in the vertical direction may be setas the threshold to be compared with the size of an object area of theobject class “train”. A size of 80 pixels in the horizontal direction×60pixels in the vertical direction may be set as the threshold to becompared with the size of an object area of the object class “flower”.

At step S306, the gaze detecting unit control unit 115 controls todisable the operation of the gaze detecting unit so that the gazedetection (operation of detecting the user's gaze) is not performed. Forexample, the gaze detecting unit control unit 115 controls to disablethe operation of at least one of the first infrared-light irradiatingunit 111, the second imaging element 120, the third preprocessing unit121, and the second detection unit 126. This reduces the powerconsumption of the image-capturing apparatus 100. Step S306 is performedwhen the gaze point is not needed to determine the AF point (thetracking target is used). As such, disabling the operation of the gazedetecting unit does not cause any problems. For example, when theprocess proceeds from step S303 to step S306, that is, when an objectarea of the partial object class “eye” is detected in only one of themultiple object areas of the object class having the highest trackingpriority, the central control unit 122 determines the position of theobject area of the partial object class “eye” (tracking result) as theAF point. Thus, even if the operation of the gaze detecting unit isdisabled, autofocusing on the object is performed in a suitable manner(with normal performance).

At step S307, the gaze detecting unit control unit 115 controls toenable the operation of the gaze detecting unit so as to perform gazedetection. For example, the gaze detecting unit control unit 115controls to set the operation cycles of the first infrared-lightirradiating unit 111, the second imaging element 120, the thirdpreprocessing unit 121, and the second detection unit 126 topredetermined operation cycles. Step S307 is performed when the gazepoint is used to determine the AF point. As such, the operation of thegaze detecting unit needs to be enabled. In one example, the process mayproceed from step S311 to step S307, that is, the size of the focusedobject area is greater than or equal to the predetermined threshold. Inthis case, when the gaze point is present in the focused object area,the central control unit 122 determines the gaze point as the AF point.Thus, although the power consumption of the image-capturing apparatus100 increases, the gaze detecting unit is enabled so that autofocusingis performed in a suitable manner.

Step S306 may be performed when the possibility of using the gaze pointis low. Step S306 may also be performed when the gaze point is not usedto determine the AF point but is used by other operations. Whenperforming step S306, the gaze detecting unit may be used as ageneral-purpose input apparatus. For example, when performing step S306,icons of different image-capturing modes may be displayed on the displayunit 118, and a change of the image-capturing mode may be determinedbased on the gaze point and the gazing time. In this case, the gazedetecting unit cannot be disabled. For this reason, at step S306, thegaze detecting unit control unit 115 may control to set the operationcycle of the gaze detecting unit to an operation cycle longer than theoperation cycle determined at step S307. For example, when step S307sets an operation cycle of 1/60 seconds (operation cycle of performing60 operations per second), step S306 may set an operation cycle of 1/30seconds (operation cycle of performing 30 operations per second). Whenstep S307 sets an operation cycle of 1/120 seconds (operation cycle ofperforming 120 operations per second), step S306 may set an operationcycle of 1/60 seconds or 1/30 seconds. The gaze detecting unit controlunit 115 may control to set the operation cycle of at least one of thefirst infrared-light irradiating unit 111, the second imaging element120, the third preprocessing unit 121, and the second detection unit 126to an operation cycle longer than the operation cycle set at the stepS307. A longer operation cycle reduces the execution frequency of theoperation (the number of times the operation is executed per unit time),thereby reducing the power consumption of the image-capturing apparatus100.

Furthermore, the gaze detecting unit control unit 115 may control theoperation of one of the first infrared-light irradiating unit 111, thesecond imaging element 120, the third preprocessing unit 121, and thesecond detection unit 126, or two or more (some or all) of them. Whenthe operation cycles of two or more of the first infrared-lightirradiating unit 111, the second imaging element 120, the thirdpreprocessing unit 121, and the second detection unit 126 arecontrolled, these operation cycles may be controlled to be the same ordifferent.

The method for controlling the operation of the gaze detecting unit isnot limited to the above method. At step S306 or step S307, the gazedetecting unit control unit 115 may change the operation cycle of thegaze detecting unit according to the object-detection result. Forexample, when three or more object areas of the object class “humanbody” are detected and also an object area of an object class relatingto sport such as “ball” or “racket” is detected, the gaze detecting unitcontrol unit 115 may determine that the user is capturing an image of asport and determine the shortest operation cycle, such as 1/240 seconds,as the operation cycle of the gaze detecting unit. When one object areaof the object class “food” is detected and the size of this object areais greater than or equal to the predetermined threshold, the gazedetecting unit control unit 115 may determine that the user is capturingan image of food and determines a long operation cycle, such as 1/15seconds, as the operation cycle of the gaze detecting unit. When oneobject area of the object class “train” is detected and the size of thisobject area is greater than or equal to the predetermined threshold, thegaze detecting unit control unit 115 may determine that the user iscapturing an image of a train and determines a short operation cycle,such as 1/60 seconds, as the operation cycle of the gaze detecting unit.

When only one object area of the object class “human body” is detectedwhile the user is capturing an image of a sport, this object area islikely to be the area of the athlete (player) on whom the user wants tofocus. Thus, when only one object area of the object class “human body”is detected, the position of the object area of the object class “humanbody” may be determined as the AF point without using the gaze point.Also, while the user is capturing an image of a sport, the gazedetecting unit control unit 115 may control the operation of the gazedetecting unit according to the number of object areas of the objectclass “human body”. When only one object area of the object class “humanbody” is detected and also an object area of an object class relating tosport such as “ball” or “racket” is detected, the position of the objectarea of the object class “human body” may be determined as the AF pointwithout using the gaze point. For this reason, the gaze detecting unitcontrol unit 115 may disable the operation of the gaze detecting unit,or may lengthen the operation cycle of the gaze detecting unit. Whenmultiple object areas of the object class “human body” are detected andan object area of an object class relating to sport is detected, it isnot easy to determine on which one of the multiple object areas of theobject class “human body” the user wants to focus. For this reason, thegaze detecting unit control unit 115 may enable the operation of thegaze detecting unit, or may shorten the operation cycle of the gazedetecting unit.

FIG. 4 shows an example of images captured by the first imaging element102 and displayed on the display unit 118, gaze points, AF points, andthe like. Referring to FIG. 4 , a specific example of the operation ofFIG. 3 is now described. The following description is directed to a casein which step S306 and step S307 are switched depending on the size ofan object area.

An image 400 is an image captured by the first imaging element 102 anddisplayed on the display unit 118. The image 400 shows a potted flower401. The first detection unit 112 detects an object area 404circumscribing the potted flower 401 from the image 400 stored in thetracking memory 110. The first detection unit 112 generates informationabout the object area 404, including the object class “flower” and thesize of the object area 404 (40 pixels in the horizontal direction×40pixels in the vertical direction). Since the object class “flower” isnot a specific object class (predetermined candidate for tracking), theprocess proceeds from step S301 to step S312. Also, since the totalnumber of detected object areas is one, the process proceeds from stepS312 to step S311. In this example, the threshold (threshold to becompared with the size of the object area) used at step S311 is set to asize of 80 pixels in the horizontal direction×60 pixels in the verticaldirection. Since the size of the object area 404 (40 pixels in thehorizontal direction×40 pixels in the vertical direction) is less thanthe threshold, the process proceeds to step S306, so that the operationof the gaze detecting unit is disabled. Since the operation of the gazedetecting unit is disabled, the central control unit 122 determines thecenter position 403 of the object area 404 as the AF point, instead ofthe gaze point 402. Also, since the operation of the gaze detecting unitis disabled, the gaze point 402 is neither detected nor displayed.

An image 410 is an image displayed by the display unit 118 on a frame(timing) different from that of the image 400. The image 410 shows apotted flower 411. The first detection unit 112 detects an object area414 circumscribing the potted flower 411 from the image 410 stored inthe tracking memory 110. The first detection unit 112 generatesinformation about the object area 414, including the object class“flower” and the size of the object area 414 (90 pixels in thehorizontal direction×90 pixels in the vertical direction). As with theimage 400, the process proceeds to step S311. Since the size of theobject area 414 (90 pixels in the horizontal direction×90 pixels in thevertical direction) is greater than or equal to the threshold (80 pixelsin the horizontal direction×60 pixels in the vertical direction), theprocess proceeds to step S307, so that the operation of the gazedetecting unit is enabled. Since the operation of the gaze detectingunit is enabled, the central control unit 122 determines a position 413that is the same as the gaze point 412 as the AF point, instead of thecenter position of the object area 414.

In this manner, when the size of the object area is greater than orequal to the predetermined threshold, the gaze detecting unit is enabledand the gaze point is determined as the AF point, so that theautofocusing is performed at the AF point that is desired by the user.Furthermore, when the size of the object area is less than thepredetermined threshold, the power consumption of the image-capturingapparatus 100 can be reduced by disabling the gaze detecting unit. Whenthe object area (object) is small, in many cases, selecting any positionin the object area as the AF point will result in substantially the sameimage being captured after autofocusing. Accordingly, even if the gazepoint is not determined as the AF point (even if the center position ofthe object area is determined as the AF point), the autofocusing can besubstantially performed as intended by the user.

As described above, based on the object-detection result, the firstembodiment controls the operation of the gaze detecting unit to beenabled or disabled, or controls the operation cycle of the gazedetecting unit. This allows gaze detection to be enabled or acceleratedonly when needed. In other words, gaze detection can be disabled orslowed down when not needed. As a result, the image-capturing apparatus100 consumes less power while limiting a decrease in the accuracy of theprocess that uses gaze detection. The first infrared-light irradiatingunit 111 of the gaze detecting unit emits light and thus consumesrelatively large power, and the second imaging element 120 of the gazedetecting unit captures images and thus consumes relatively large power.Accordingly, a significant effect can be expected in terms of thereduction of the power consumption of the gaze detecting unit.

Examples of situations that require gaze detection are as follows.

-   -   A fast-moving object (candidate for tracking) is detected.    -   The size of the object is large, and the AF point should be        selected inside the object.    -   A combination of objects, such as a person and a ball, is        detected that suggests that the user is capturing (or about to        capture) an image of a sport.    -   Multiple objects are detected.    -   No object is detected.

Examples of situations that do not require gaze detection are asfollows.

-   -   Only one object of a specific type is detected, such as an eye        or the helmet of an occupant of a motorcycle, and the position        to be determined as the AF point is clear.    -   The size of the object is small, and the position to be        determined as the AF point is clear.

According to the first embodiment, the following operations are possiblewhen capturing an image of a train. When the train approaches from adistance, the gaze detection is disabled and the center position of thetrain is determined as the AF point because the size of the train in thecaptured image is less than the threshold. When the train approaches andthe size of the train in the images greater than or equal to thethreshold, the gaze detection is enabled, and the gaze point isdetermined as the AF point. Then, when the train moves away and the sizeof the train in the image becomes less than the threshold, the gazedetection is disabled, and the center position of the train isdetermined as the AF point.

Second Embodiment

A second embodiment of the present invention is now described. In thefollowing, the items (such as configuration and processing) that are thesame as those in the first embodiment are not described, and the itemsdifferent from the first embodiment are described. In the secondembodiment, the gaze detecting unit control unit 115 controls theoperation of the gaze detecting unit based on the object-detectionresult only when the image-capturing apparatus 100 is set to a specificmode.

FIG. 5 is a flowchart showing an example of an operation of theimage-capturing apparatus 100 according to the second embodiment. FIG. 5shows an example of an operation of controlling the gaze detecting unitbased on the image-capturing mode of the image-capturing apparatus 100.

At step S501, the gaze detecting unit control unit 115 determineswhether the image-capturing mode of the image-capturing apparatus 100 isan AF mode for capturing non-moving objects. The gaze detecting unitcontrol unit 115 proceeds to step S502 when the image-capturing mode ofthe image-capturing apparatus 100 is not the AF mode for capturingnon-moving objects, and proceeds to step S505 when the image-capturingmode of the image-capturing apparatus 100 is the AF mode for capturingnon-moving objects.

At step S502, the gaze detecting unit control unit 115 determineswhether the image-capturing mode of the image-capturing apparatus 100 isan AF mode for capturing moving objects. The gaze detecting unit controlunit 115 proceeds to step S503 when the image-capturing mode of theimage-capturing apparatus 100 is not the AF mode for capturing movingobjects, and proceeds to step S504 when the image-capturing mode of theimage-capturing apparatus 100 is the AF mode for capturing movingobjects. In the second embodiment, it is assumed that the AF mode forcapturing non-moving objects, the AF mode for capturing moving objects,or an automatic determination AF mode is set as the AF mode.Accordingly, the process proceeds to step S503 when the image-capturingmode of the image-capturing apparatus 100 is the automatic determinationAF mode. The automatic determination AF mode is an AF mode in which thedetermination on a moving object or a non-moving object is performedautomatically for image capturing.

At step S503, the gaze detecting unit control unit 115 controls theoperation of the gaze detecting unit based on the object-detectionresult. For example, the gaze detecting unit control unit 115 controlsthe gaze detecting unit through the operation shown in FIG. 3 .

At step S504, the gaze detecting unit control unit 115 enables theoperation of the gaze detecting unit to capture an image of a movingobject, and fixes the operation cycle of the gaze detecting unit to theshortest operation cycle (for example, 1/240 seconds). The operation ofthe gaze detecting unit is not controlled based on the object-detectionresult.

At step S505, the gaze detecting unit control unit 115 enables theoperation of the gaze detecting unit to capture an image of a non-movingobject, and fixes the operation cycle of the gaze detecting unit to thelongest operation cycle (for example, ⅕ seconds). The operation of thegaze detecting unit is not controlled based on the object-detectionresult.

As described above, according to the second embodiment, the operation ofthe gaze detecting unit is controlled based on the object-detectionresult only in a specific mode. This reduces the power consumption ofthe image-capturing apparatus 100 and improves the convenience of theimage-capturing apparatus 100.

Third Embodiment

A third embodiment of the present invention is now described. In thefollowing, the items (such as configuration and processing) that are thesame as those in the first embodiment are not described, and the itemsdifferent from the first embodiment are described. The infrared lightemitted by the first infrared-light irradiating unit 111 to detect agaze may reduce the accuracy of eye-proximity detection. The infraredlight emitted by the second infrared-light irradiating unit 129 to senseeye proximity may reduce the accuracy of gaze detection. For example,any interference between the infrared light emitted by the firstinfrared-light irradiating unit 111 and the infrared light emitted bythe second infrared-light irradiating unit 129 may reduce the accuracyof gaze detection and eye-proximity detection. For this reason, thethird embodiment controls the state (operation) of the eye-proximitysensing unit 128 according to the state of the gaze detecting unit.

FIG. 6 is a flowchart showing an example of an operation of theimage-capturing apparatus 100 according to the third embodiment. FIG. 6shows an example of an operation of controlling the state (operation) ofthe eye-proximity sensing unit 128 according to the state of the gazedetecting unit.

At step S601, the gaze detecting unit control unit 115 determineswhether the operation of the gaze detecting unit (e.g., the firstinfrared-light irradiating unit 111 and the second imaging element 120)is enabled. The gaze detecting unit control unit 115 proceeds to stepS602 when the operation of the gaze detecting unit is not enabled(disabled), and proceeds to step S603 when the operation of the gazedetecting unit is enabled.

At step S602, since the operation of the gaze detecting unit is notenabled, the gaze detecting unit control unit 115 determines that theoperation of the eye-proximity sensing unit 128 (the secondinfrared-light irradiating unit 129 and the infrared-light sensing unit131) can be freely controlled. For example, since the second imagingelement 120 does not capture the infrared light emitted by the secondinfrared-light irradiating unit 129, the gaze detecting unit controlunit 115 determines that the second infrared-light irradiating unit 129is allowed to emit infrared light. Also, since the first infrared-lightirradiating unit 111 does not emit infrared light, which would otherwisebe incident on the infrared-light sensing unit 131, the gaze detectingunit control unit 115 determines that the infrared-light sensing unit131 is allowed to receive incident infrared light. As such, the gazedetecting unit control unit 115 enables the operation of theeye-proximity sensing unit 128. A shorter operation cycle of theeye-proximity sensing unit 128 increases the responsiveness in sensingeye proximity. Thus, the gaze detecting unit control unit 115 preferablyshortens the operation cycle of the eye-proximity sensing unit 128. Forexample, when the shortest operation cycle of the eye-proximity sensingunit 128 is 1/120 seconds, the gaze detecting unit control unit 115 mayset the operation cycle of the eye-proximity sensing unit 128 to 1/120seconds.

At step S603, the gaze detecting unit control unit 115 determineswhether the operation cycle of the gaze detecting unit is apredetermined operation cycle (first operation cycle). For example, thefirst operation cycle is the operation cycle used when the operationshown in FIG. 3 is not performed. The operation of FIG. 3 may result inthe operation cycle of the gaze detecting unit being set to an operationcycle that is longer or shorter than the first operation cycle. The gazedetecting unit control unit 115 proceeds to step S604 when the operationcycle of the gaze detecting unit is the first operation cycle, andproceeds to step S605 when the operation cycle of the gaze detectingunit is not the first operation cycle.

At step S604, the gaze detecting unit control unit 115 controls to setthe operation cycle of the eye-proximity sensing unit 128 to apredetermined operation cycle (second operation cycle). The secondoperation cycle, which is the operation cycle of the eye-proximitysensing unit 128, is determined based on the first operation cycle,which is the operation cycle of the gaze detecting unit.

FIG. 7 shows an example of the operation of the first infrared-lightirradiating unit 111 in the first operation cycle and the operation ofthe second infrared-light irradiating unit 129 in the second operationcycle. The time from timing T701 to timing T703 and the time from timingT703 to timing T705 are both 1/60 seconds. The time from timing T705 totiming T707 and the time from timing T707 to timing T709 are both 1/60seconds.

In this example, the first infrared-light irradiating unit 111 operatesas follows. In the following operation, the first infrared-lightirradiating unit 111 is turned on (infrared-light irradiation) andturned off (infrared-light non-irradiation) once every 1/30 seconds.That is, the operation cycle of the first infrared-light irradiatingunit 111 is set to 1/30 seconds (first operation cycle).

-   -   Infrared light is emitted from timing T701 to timing T702        (infrared-light irradiation 710).    -   Infrared light is not emitted from timing T702 to timing T705        (infrared-light non-irradiation 711).    -   Infrared light is emitted from timing T705 to timing T706        (infrared-light irradiation 712).    -   Infrared light is not emitted from timing T706 to timing T709        (infrared-light non-irradiation 713).

In accordance with the above operation of the first infrared-lightirradiating unit 111, the second infrared-light irradiating unit 129operates as follows. In the following operation, the secondinfrared-light irradiating unit 129 is turned on (infrared-lightirradiation) and turned off (infrared-light non-irradiation) once every1/30 seconds. That is, the operation cycle of the second infrared-lightirradiating unit 129 is also set to 1/30 seconds (second operationcycle).

-   -   Infrared light is not emitted from timing T701 to timing T703        (infrared-light non-irradiation 720).    -   Infrared light is emitted from timing T703 to timing T704        (infrared-light irradiation 721).    -   Infrared light is not emitted from timing T704 to timing T707        (infrared-light non-irradiation 722).    -   Infrared light is emitted from timing T707 to timing T708        (infrared-light irradiation 723).    -   Infrared light is not emitted from timing T708 to timing T709        (infrared-light non-irradiation 724).

As described above, in the example of FIG. 7 , both the firstinfrared-light irradiating unit 111 and the second infrared-lightirradiating unit 129 operate with an operation cycle of 1/30 seconds.However, to reduce (or eliminate) the duration in which both the firstinfrared-light irradiating unit 111 and the second infrared-lightirradiating unit 129 are turned on, the periods of infrared-lightirradiation 721 and 723 of the second infrared-light irradiating unit129 do not coincide with the periods of infrared-light irradiation 710and 712 of the first infrared-light irradiating unit 111. Separating theperiods of infrared-light irradiation of the second infrared-lightirradiating unit 129 from the periods of infrared-light irradiation ofthe first infrared-light irradiating unit 111 reduces the influence ofthe infrared light of the first infrared-light irradiating unit 111 oneye-proximity detection and the influence of the infrared light of thesecond infrared-light irradiating unit 129 on gaze detection.

It should be noted that FIG. 7 is an example, and the first and secondinfrared-light irradiating units 111 and 129 may be controlled in anymanner as long as the periods of infrared-light irradiation of the firstinfrared-light irradiating unit 111 do not coincide with the periods ofinfrared-light irradiation of the second infrared-light irradiating unit129. The first operation cycle may be longer or shorter than 1/30seconds, the second operation cycle may be longer or shorter than 1/30seconds, the second operation cycle may be the same as the firstoperation cycle, and the second operation cycle may be different fromthe first operation cycle. For example, by tolerating a decrease in theresponsiveness of eye-proximity detection, the operation cycle (secondoperation cycle) of the eye-proximity sensing unit 128 may be set to1/15 fps to reduce the power consumption of the eye-proximity sensingunit 128. When lengthening the operation cycle of the eye-proximitysensing unit 128, the duration of each infrared-light non-irradiationmay be lengthened without changing the duration of each infrared-lightirradiation. Thus, when the operation cycle of the gaze detecting unitis constant, a longer operation cycle of the eye-proximity sensing unit128 facilitates the separation of the periods of infrared-lightirradiation of the second infrared-light irradiating unit 129 from theperiods of infrared-light irradiation of the first infrared-lightirradiating unit 111, and also facilitates the limitation of a decreasein the accuracy of gaze detection and eye-proximity detection.

At step S605, the gaze detecting unit control unit 115 determineswhether the operation cycle of the gaze detecting unit is set to anoperation cycle that is longer than the first operation cycle. When theoperation cycle of the gaze detecting unit is set to an operation cyclethat is longer than the first operation cycle, the gaze detecting unitcontrol unit 115 proceeds to step S606. When the operation cycle of thegaze detecting unit is not set to an operation cycle longer than thefirst operation cycle (specifically, it is set to an operation cycleshorter than the first operation cycle), the gaze detecting unit controlunit 115 proceeds to step S607.

At step S606, since the operation cycle of the gaze detecting unit islonger than the first operation cycle, the gaze detecting unit controlunit 115 sets the operation cycle of the eye-proximity sensing unit 128to be shorter than the second operation cycle. When the operation cycleof the first infrared-light irradiating unit 111 is long, the durationof infrared-light non-irradiation of the first infrared-lightirradiating unit 111 (the time between successive infrared-lightirradiations) is long. In this case, it is easy to separate the periodsof infrared-light irradiation of the second infrared-light irradiatingunit 129 from the periods of infrared-light irradiation of the firstinfrared-light irradiating unit 111. As such, the gaze detecting unitcontrol unit 115 sets the operation cycle of the eye-proximity sensingunit 128 to be shorter than the second operation cycle to increase theresponsiveness of eye-proximity detection.

At step S607, since the operation cycle of the gaze detecting unit isshorter than the first operation cycle, the gaze detecting unit controlunit 115 sets the operation cycle of the eye-proximity sensing unit 128to be longer than the second operation cycle. When the operation cycleof the first infrared-light irradiating unit 111 is short, the durationof infrared-light non-irradiation of the first infrared-lightirradiating unit 111 (the time between successive infrared-lightirradiations) is short. In this case, it is more difficult to separatethe periods of infrared-light irradiation of the second infrared-lightirradiating unit 129 from the periods of infrared-light irradiation ofthe first infrared-light irradiating unit 111. For this reason, the gazedetecting unit control unit 115 sets the operation cycle of theeye-proximity sensing unit 128 to be longer than the second operationcycle and separates the periods of infrared-light irradiation of thesecond infrared-light irradiating unit 129 from the periods ofinfrared-light irradiation of the first infrared-light irradiating unit111.

As described above, according to the third embodiment, the state(operation) of the eye-proximity sensing unit 128 is controlledaccording to the state of the gaze detecting unit. This reduces theinfluence of the infrared light of the first infrared-light irradiatingunit 111 on eye-proximity detection and the influence of the infraredlight of the second infrared-light irradiating unit 129 on gazedetection, thereby limiting a decrease in the accuracy of gaze detectionand eye-proximity detection.

Fourth Embodiment

A fourth embodiment of the present invention is now described. In thefollowing, the items (such as configuration and processing) that are thesame as those in the first embodiment are not described, and the itemsdifferent from the first embodiment are described. The fourth embodimentcontrols the operation of the gaze detecting unit based on the remainingbattery level of the image-capturing apparatus.

FIG. 8 is a block diagram schematically showing an example of theinternal structure of an image-capturing apparatus 800 according to thefourth embodiment. The image-capturing apparatus 800 has a configurationin which a battery 802 and a remaining-battery-level determination unit801 are added to the image-capturing apparatus 100 shown in FIG. 1 . Thebattery 802 supplies electric power to the components of theimage-capturing apparatus 800 to drive the components. Theremaining-battery-level determination unit 801 determines whether theimage-capturing apparatus 800 (battery 802) is in a low battery state,and outputs the determination result (remaining-battery-leveldetermination result) to the gaze detecting unit control unit 115. Thelow battery state is a state in which the remaining battery level of thebattery 802 is less than or equal to a predetermined threshold. The gazedetecting unit control unit 115 controls the operation of the gazedetecting unit based on the remaining-battery-level determinationresult.

FIG. 9 is a flowchart showing an example of an operation of theimage-capturing apparatus 800. FIG. 9 shows an example of an operationof controlling the gaze detecting unit based on aremaining-battery-level determination result. The gaze detecting unit isassumed to be operating with a predetermined operation cycle at thestart of the operation of FIG. 9 .

At step S901, the gaze detecting unit control unit 115 determineswhether the remaining-battery-level determination result obtained fromthe remaining-battery-level determination unit 801 indicates the lowbattery state. The gaze detecting unit control unit 115 proceeds to stepS902 when the remaining-battery-level determination result does notindicate the low battery state, and proceeds to step S903 when theremaining-battery-level determination result indicates the low batterystate.

At step S902, since the battery 802 is not in the low battery state, thegaze detecting unit control unit 115 does not change the operation cycleof the gaze detecting unit from the predetermined operation cycle.

At step S903, since the battery 802 is in the low battery state, thegaze detecting unit control unit 115 controls the operation of the gazedetecting unit. For example, the gaze detecting unit control unit 115controls the gaze detecting unit through the operation shown in FIG. 3 .

The operation of FIG. 9 may be combined with the operations of FIGS. 5and 6 . For example, the operation of FIG. 5 is not performed when thebattery 802 is not in the low battery state, and the operation of FIG. 5may be performed when the battery 802 is in the low battery state.Alternatively, the operation of FIG. 5 may be performed when the battery802 is not in the low battery state, and the operation of FIG. 3 isperformed when the battery 802 is in the low battery state.

Furthermore, instead of switching between presence and absence of thecontrol of operation of the gaze detecting unit according to whether thebattery 802 is in the low battery state, the method for controlling theoperation of the gaze detecting unit may be switched. For example, thegaze detecting unit may be controlled through the operation of FIG. 3regardless of whether the battery 802 is in the low battery state, butstep S311 in FIG. 3 may be skipped and the process proceeds to step S307when the battery 802 is in the low battery state.

As described above, the fourth embodiment controls the operation of thegaze detecting unit based on the result of determination on whether thebattery 802 is in the low battery state. For example, when the battery802 is not in the low battery state, the operation cycle of the gazedetecting unit may be set to the predetermined operation cycle. When thebattery 802 is in the low battery state, the operation of the gazedetecting unit may be controlled based on the object-detection result.As a result, when the remaining battery level is low, the operation ofthe gaze detecting unit is disabled or slowed down based on theobject-detection result to reduce the power consumption of theimage-capturing apparatus 800. This increases the duration in which theimage-capturing apparatus 800 can capture images. When the remainingbattery level is high, the gaze detecting unit can be operated at highspeed with a short operation cycle, improving the operability of variousoperations that use the gaze.

It should be noted that the first to fourth embodiments are merelyexamples, and the present invention also encompasses configurationsobtained by appropriately modifying or changing the configurations ofthe first to fourth embodiments within the scope of the presentinvention. The present invention also includes configurations obtainedby appropriately combining configurations of the first to fourthembodiments.

For example, in the examples described above, the present invention isapplied to an image-capturing apparatus having a structure of amirrorless single-lens camera, but the image-capturing apparatus towhich the present invention is applicable is not limited to a mirrorlesssingle-lens camera and may be other apparatuses such as a single-lensreflex camera. That is, the user may look into the finder and see theoptical image of the object itself, instead of the image displayed onthe display unit.

The image-capturing apparatus of the present invention may be asmartphone, a tablet terminal, or the like. For example, a smartphonecaptures an image with a rear camera provided on the back of thesmartphone, and displays the image captured by the rear camera on adisplay unit provided on the front of the smartphone. Additionally, thesmartphone detects the user's gaze using a front camera provided on thefront of the smartphone. Then, the smartphone detects the object fromthe image captured by the rear camera, and controls the operation of thefront camera and the like based on the detection result.

The image-capturing apparatus of the present invention may be aneyeglasses-type (goggles-type) wearable terminal. For example, aneyeglasses-type wearable terminal may include a camera that captures animage of the viewing area of the user wearing the terminal and a gazedetection sensor that detects the gaze of the user. The eyeglasses-typewearable terminal detects an object from the image captured by thecamera, and controls the operation of the gaze detection sensor or thelike based on the detection result. Such a wearable terminal may useaugmented reality (AR) technology, for example. That is, the presentinvention is applicable to electronic apparatuses (e.g., AR glasses)that use the AR technology. Similarly, the present invention isapplicable to electronic apparatuses that use other XR technologies suchas virtual reality (VR) technology, mixed reality (MR) technology, andsubstitutional reality (SR) technology.

According to the present disclosure, it is possible to reduce the powerconsumption of the image-capturing apparatus capable of detecting thegaze of the user.

OTHER EMBODIMENTS

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2021-113636, filed on Jul. 8, 2021, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image-capturing apparatus comprising: an imagesensor; a gaze detecting sensor configured to detect a gaze of a user;and at least one memory and at least one processor which function as: anobject-detecting unit configured to detect an object from an imagecaptured by the image sensor; and a control unit configured to, if theobject-detecting unit detects an object of a specific type in a casewhere a state of the gaze detecting sensor is a first state fordetecting the gaze, change the state of the gaze detecting sensor to asecond state in which an electric power consumption of the gazedetecting sensor is less than in the first state.
 2. The image-capturingapparatus according to claim 1, wherein the gaze detecting sensorincludes at least one of an irradiation member configured to irradiatethe user with infrared light, a second image sensor configured tocapture an image of the user, and a detector configured to detect thegaze of the user based on the image captured by the second image sensor.3. The image-capturing apparatus according to claim 1, wherein theobject of the specific type is an object which is a candidate fortracking.
 4. The image-capturing apparatus according to claim 1, whereinthe object of the specific type is an movable object.
 5. Theimage-capturing apparatus according to claim 1, wherein the object ofthe specific type includes at least one of a living being of a mammal, aliving being of a bird, a living being of a reptile, a living being of afish, a living being of a crustacean, an insect, and a vehicle.
 6. Theimage-capturing apparatus according to claim 1, wherein the object ofthe specific type includes at least one of a human body, a cat, a dog,and a horse.
 7. The image-capturing apparatus according to claim 1,wherein the object of the specific type includes at least one of arailroad vehicle, an automobile, an aircraft, and a ship.
 8. Theimage-capturing apparatus according to claim 1, wherein the second stateis a state in which an operation of the gaze detecting sensor isstopped.
 9. The image-capturing apparatus according to claim 1, whereinthe second state is a state in which an operation cycle of the gazedetecting sensor is longer than in the first state.
 10. Theimage-capturing apparatus according to claim 1, wherein theobject-detecting unit detects, from the image, an object and a partialobject that is a part of the object.
 11. The image-capturing apparatusaccording to claim 10, wherein in a case where the state of the gazedetecting sensor is the first state, even if the object-detecting unitdetects an object of the specific type, if the object-detecting unitdoes not detect a partial object of the object, the control unit doesnot change the state of the gaze detecting sensor.
 12. Theimage-capturing apparatus according to claim 11, wherein in a case wherethe state of the gaze detecting sensor is the first state and theobject-detecting unit detects an object of the specific type, even ifthe object-detecting unit does not detect a partial object of theobject, if a size of the object is smaller than a threshold size, thecontrol unit, changes the state of the gaze detecting sensor to thesecond state.
 13. The image-capturing apparatus according to claim 11,wherein each of a plurality of types that differ in priority ispredetermined as the specific type, and in a case where the state of thegaze detecting sensor is the first state and the object-detecting unitdetects objects of a plurality of specific types, the control unitchanges the state of the gaze detecting sensor based on a result ofdetection by the object-detecting unit with regard to one specific typeof the specific types that has the highest priority.
 14. Theimage-capturing apparatus according to claim 10, wherein in a case wherethe state of the gaze detecting sensor is the first state, even if theobject-detecting unit detects an object of the specific type, if theobject-detecting unit detects a plurality of objects of a same type asobjects of the specific type and detects a plurality of partial objectsof a same type respectively corresponding to the plurality of objects,the control unit does not change the state of the gaze detecting sensor.15. The image-capturing apparatus according to claim 10, wherein in acase where the state of the gaze detecting sensor is the first state andthe object-detecting unit detects a plurality of objects of a same typeas objects of the specific type, if a partial object of a first type isdetected in only one of the plurality of objects, the control unitchanges the state of the gaze detecting sensor to the second state, evenif partial objects of the first type are detected in two or more of theobjects, if a partial object of a second type is detected in only one ofthe plurality of objects, the control unit changes the state of the gazedetecting sensor to the second state, even if partial objects of thefirst type are detected in two or more of the objects and partialobjects of the second type are detected in two or more of the objects,if a partial object of a third type is detected in only one of theplurality of objects, the control unit changes the state of the gazedetecting sensor to the second state, and if partial objects of thefirst type are detected in two or more of the objects, partial objectsof the second type are detected in two or more of the objects, andpartial objects of the third type are detected in two or more of theobjects of the plurality of objects, the control unit does not changethe state of the gaze detecting sensor.
 16. The image-capturingapparatus according to claim 10, wherein the partial object includes aneye.
 17. The image-capturing apparatus according to claim 10, whereinthe partial object includes a head.
 18. The image-capturing apparatusaccording to claim 10, wherein the partial object includes a face. 19.The image-capturing apparatus according to claim 1, wherein in a casewhere the state of the gaze detecting sensor is the first state, even ifthe object-detecting unit does not detect an object of the specifictype, if the object-detecting unit detects an object having a sizesmaller than a threshold size, the control unit changes the state of thegaze detecting sensor to the second state.
 20. The image-capturingapparatus according to claim 19, wherein in a case where the state ofthe gaze detecting sensor is the first state, even if theobject-detecting unit does not detect an object of the specific type anddetects an object having a size smaller than the threshold size, if theobject-detecting unit detects a plurality of objects, the control unitdoes not change the state of the gaze detecting sensor.
 21. Theimage-capturing apparatus according to claim 1, further comprising: aneye-proximity sensor configured to sense a presence of an eye inproximity to a finder, wherein the control unit controls a state of theeye-proximity sensor according to the state of the gaze detectingsensor.
 22. The image-capturing apparatus according to claim 21, whereinin a case where the state of the gaze detecting sensor is the firststate, the control unit controls to set the state of the eye-proximitysensor to a third state, and in a case where the state of the gazedetecting sensor is the second state, the control unit controls to setthe state of the eye-proximity sensor to a fourth state in which anoperation cycle of the eye-proximity detection unit is shorter than inthe third state.
 23. The image-capturing apparatus according to claim 1,wherein in a case where a remaining battery level of the image-capturingapparatus is greater than a threshold level, the control unit does notchange the state of the gaze detecting sensor based on a result ofdetection by the object-detecting unit, and in a case where theremaining battery level is less than the threshold level, the controlunit changes the state of the gaze detecting sensor based on the resultof detection by the object-detecting unit.
 24. The image-capturingapparatus according to claim 1, wherein in a case where theimage-capturing apparatus is not set to a specific mode, the controlunit does not change the state of the gaze detecting sensor based on aresult of detection by the object-detecting unit, and in a case wherethe image-capturing apparatus is set to the specific mode, the controlunit changes the state of the gaze detecting sensor based on the resultof detection by the object-detecting unit.
 25. An image-capturingapparatus comprising: an image sensor; a gaze detecting sensorconfigured to detect a gaze of a user; and at least one memory and atleast one processor which function as: an object-detecting unitconfigured to detect an object from an image captured by the imagesensor; and a control unit configured, in a state of the gaze detectingsensor is a first state for detecting the gaze: to not change the stateof the gaze detecting sensor even if the object-detecting unit detects aplurality of human bodies and at least one ball; and to change the stateof the gaze detecting sensor to a second state in which an electricpower consumption of the gaze detecting sensor is less than in the firststate if the object-detecting unit detects one human body and at leastone ball.
 26. A control method of an image-capturing apparatus includingan image sensor and a gaze detecting sensor configured to detect a gazeof a user, the control method comprising: a step of detecting an objectfrom an image captured by the image sensor; and a step of, if an objectof a specific type is detected from the image in a case where a state ofthe gaze detecting sensor is a first state for detecting the gaze,changing the state of the gaze detecting sensor to a second state inwhich an electric power consumption of the gaze detecting sensor is lessthan in the first state.
 27. A control method of an image-capturingapparatus including an image sensor and a gaze detecting sensorconfigured to detect a gaze of a user, the control method comprising: astep of detecting an object from an image captured by the image sensor;and a step of, in a state of the gaze detecting sensor is a first statefor detecting the gaze: not changing the state of the gaze detectingsensor even if a plurality of human bodies and at least one ball aredetected from the image; and changing the state of the gaze detectingsensor to a second state in which an electric power consumption of thegaze detecting sensor is less than in the first state if one human bodyand at least one ball are detected from the image.
 28. A non-transitorycomputer readable medium that stores a program, wherein the programcauses a computer to execute a control method of an image-capturingapparatus including an image sensor and a gaze detecting sensorconfigured to detect a gaze of a user, the control method comprising: astep of detecting an object from an image captured by the image sensor;and a step of, if an object of a specific type is detected from theimage in a case where a state of the gaze detecting sensor is a firststate for detecting the gaze, changing the state of the gaze detectingsensor to a second state in which an electric power consumption of thegaze detecting sensor is less than in the first state.
 29. Anon-transitory computer readable medium that stores a program, whereinthe program causes a computer to execute a control method of animage-capturing apparatus including an image sensor and a gaze detectingsensor configured to detect a gaze of a user, the control methodcomprising: a step of detecting an object from an image captured by theimage sensor; and a step of, in a state of the gaze detecting sensor isa first state for detecting the gaze: not changing the state of the gazedetecting sensor even if a plurality of human bodies and at least oneball are detected from the image; and changing the state of the gazedetecting sensor to a second state in which an electric powerconsumption of the gaze detecting sensor is less than in the first stateif one human body and at least one ball are detected from the image.